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Abstract. This study utilized the SEC 
(Survey of Enacted Curriculum) method 

to examine the alignment between 
Chinese high school chemistry curriculum 
standards (HSCCS) and the assessment 

of ‘Chemical Reaction Principles’ in the 
National College Entrance Examinations 
(NCEEs). The HSCCS and NCEEs were coded 
into two-dimensional matrices separately 
using SPSS, MATLAB, and EXCEL. The 
alignment coefficients were analyzed both 
macroscopically and specifically based 

on two dimensions: themes and cognitive 
levels. The findings indicated a generally 
low alignment between NCEEs and HSCCS 
in the ‘Chemical Reaction Principles’ 
domain, and no statistically significant 
alignment was observed. Comparing Porter 
alignment coefficients revealed a gradual 
increase in the overall alignment level 
between 2018-2022 NCEEs and HSCCS 
due to curricular reforms. Further specific 
analyses and comparisons highlighted 
significant discrepancies between NCEEs 
and HSCCS concerning themes and 
cognitive levels. Regarding themes, ‘lonic 
Reactions and Equilibrium in Aqueous 
Solutions’ showed substantial alignment 
between NCEEs and HSCCS. However, 

for ‘Application of lonic Reactions and 
Equilibrium’ and ‘Systems and Energy, 
NCEEs diverged significantly from or 
exceeded HSCCS requirements. Concerning 
cognitive levels, NCEEs demanded higher 
levels of student cognition compared to 
HSCCS. 
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Introduction 


National College Entrance Examinations (NCEEs) hold immense impor- 
tance in China as a standardized examination determining an individual’s 
eligibility for higher education institutions (Bai et al., 2014; Zivin et al., 2020). 
As a crucial link between primary and higher education, it plays a vital role 
in improving educational quality, driving national progress, enabling social 
mobility, and contributing to the country’s modernization (Niu & Liu, 2022; 
Tian, et al., 2024). Consequently, assessing the NCEEs’ quality becomes es- 
sential to facilitate the reform of its design and enhance its nurturing func- 
tion effectively. 

In the current context of curriculum reform, standards have become 
crucial benchmarks for curriculum management and evaluation. They 
provide a foundation for measuring teaching and learning, serve as a valu- 
able reference for assessment development, and act as a critical model for 
developing teaching materials (Ministry of Education of China, 2020; Yu et al., 
2022). In essence, standards establish precise requirements and expectations 
that guide classroom instruction and various forms of assessment (Misfeldt 
et al., 2019; Troia et al., 2018). Enhancing the alignment between essential 
components of the education system, such as assessment and standards, 
is paramount to ensuring the quality of education, attaining desired objec- 
tives, and facilitating the smooth progression of curriculum reform (Newton 
& Kasten, 2013; Paik et al., 2011; Qhibi et al., 2020). 

The knowledge of ‘Chemical Reaction Principles’ holds utmost impor- 
tance in upper-secondary school chemistry education (Ministry of Educa- 
tion of China, 2019, 2020). This essential component introduces students to 
the fundamental principles and methods of studying chemical reactions, 
encompassing topics such as energy transformations, the direction, limits, 
and rates of chemical reactions, and the behavior of substances in aque- 
ous solutions (Harrison & De, 2005, Tyson et al., 1999). These foundational 
principles are crucial for students to develop a profound understanding of 
the nature and laws governing chemical reactions (Balci, 2006; Tyson et al., 
1999). Moreover, they not only form the core of basic chemistry knowledge 
but also serve as an indispensable guide for students’ future scientific re- 
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search and technological innovation (Ministry of Education of China, 2020). However, in the researcher's teaching 
practice through unstructured interviews, it was found that students had many difficulties in learning the ‘Chemi- 
cal Reaction Principles, and teachers also found it challenging to teach this part of the subject and to grasp the 
requirements of exams and standards. 

Reviewing the current studies, theories on the alignment between standards and assessment have been 
abundant (Bonner et al., 2018; Fulmer et al., 2018; Fulmer & Polikoff, 2014), and practice-level case studies have 
been distributed across the Writing, Science, World Language Programs, Mathematics, and various other disciplines 
(Copur-Gencturk et al., 2021; Liu et al., 2009; Liu & Fulmer, 2008; Troia et al., 2018). However, the current case stud- 
ies on alignment are mainly quantitative research on a discipline as a whole, and no studies specifically address 
the alignment between the standards and assessments in the ‘Chemical Reaction Principles’ section. Building 
upon the context above, this research investigates the alignment between the Chinese High School Chemistry 
Curriculum Standards (HSCCS) and the NCEEs, specifically in the ‘Chemical Reaction Principles’ domain. Alignment 
studies explore the overall alignment and differences in the distribution of themes and cognitive levels, analyzing 
the factors influencing their alignment. The findings of this study hold significance as they can provide teachers 
and students with scientifically supported and effective strategies for NCEE preparation. Moreover, the research 
outcomes can offer valuable insights for enhancing the quality of NCEE assessments and contribute to the ongo- 
ing progress of curriculum reform. 


Literature Review 
Curriculum Standards and Related Studies 


Starting from the 1970s, there was a significant expansion of school curricula (U.S. Department of Education 
Office of Educational Technology, 2002), leading to increased diversity and decreased uniformity. This fragmentation 
of teaching and learning activities created a pressing need for standards to address the disparities in education 
across schools (Reys et al., 2007). Consequently, countries began developing standardized curriculum standards 
and initiated education reform movements based on these standards (Klein, 1999). While educational traditions 
and reform contexts vary among countries, curriculum standards are universally regarded as ‘expected student 
learning outcomes. They aim to establish clear educational expectations defined by the state and the public, serv- 
ing as a basis for evaluating the performance of schools and other stakeholders (Klein, 1999; La Marca et al., 2000; 
Schmidt et al., 2001). 

Against this backdrop and within the context of China’s revised general high school curriculum standards in 
2020 (Ministry of Education of China, 2020), this study defines curriculum standards as guiding documents that 
outline the nature, philosophy, objectives, structure, content, academic quality, and implementation recommen- 
dations of a subject’s curriculum (La Marca et al., 2000; Schmidt et al., 2001). These standards form the foundation 
for establishing teaching objectives for each grade, assessing student performance, and developing teaching 
materials and test questions (Lu & Liu, 2012; Wei, 2019). Moreover, all references to curriculum standards in this 
research specifically pertain to the General High School Chemistry Curriculum Standards (2017 Edition Revised 
2020) (HSCCS) (Ministry of Education of China, 2020). 

In the realm of curriculum standards-based educational reform, researchers have actively delved into the subject 
of standards-based curriculum reform. Current research on curriculum standards primarily encompasses government 
documents and reports (Drake & Burns, 2004), the process of standards development (Webb, 2007), comparative 
analyses of curriculum standards across different countries (Wei & Ou, 2019), and the practical application and 
evaluation of standards within specific classroom teaching contexts (Park et al., 2019). These research endeavors col- 
lectively contribute to understanding and advancing curriculum standards and their implementation in education. 


Alignment Analysis of Curriculum Standards and Assessment 


In the education system, alignment primarily refers to the extent of coherence among various key components 
such as curriculum, teaching, and assessment (Lu & Liu, 2012; Webb, 2007; Yang, 2023). This study defines align- 
ment as the degree of correspondence between HSCCS and NCEEs. Currently, analysis related to alignment within 
the domain of standards-based curriculum focuses on four key areas (Table 1): alignment between standards and 
assessment; alignment between standards and teaching; alignment between standards and teaching materials; 
alignment between school learning and assessment, and alignment among three or more educational elements. 
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Table 1 
Alignment Related research 


Alignment Research Areas Related Studies 


Contino, 2013; Flowers et al., 2006; Lu & Liu, 2012; Martone & Sireci, 2009; Newton & 


SlaIaIdS Senses emt Kasten, 2013; Troia et al., 2018 


Standards & Teaching Kurz et al., 2010; Liu et al., 2009; Taub et al., 2020; Yang, 2023; 
Standards & Teaching Materials Polikoff, 2015; Yu et al., 2022 

School Learning & Assessment Kara & Cepni, 2011 

Three or More Educational Elements Mohamud & Fleck, 2010; Roach et al., 2008 


Among these areas, the alignment between assessment and standards has been extensively studied. Research 
in this domain has primarily focused on examining the alignment between the two, employing quantitative con- 
gruence methods to estimate the match between assessments and state content standards in different regions 
(Flowers et al., 2006). Other studies have analyzed the alignment between physics curriculum standards and 
teacher instructional content in countries such as China and Singapore (Liu et al., 2009). Additionally, researchers 
have explored the relationship between the level of congruence between (assessment + standards) and student 
learning outcomes. Troia et al. (2018) employed text content coding to examine the congruence between US state 
English writing standards and assessments, exploring measures of content range, frequency, balance, cognitive 
complexity, and their correlation with student writing achievement. Discussions on common methods for con- 
ducting congruence analyses between assessment and standards have also been prevalent. Martone and Sireci 
(2009) describe the three most commonly used methods, highlight their strengths and limitations, and provide 
examples of their applications. 


Theoretical Framework of Alignment Analysis: SEC Alignment Method 


Current research related to alignment analysis in the curriculum domain has primarily employed three models: 
Webb (Webb, 1999), SEC (Blank et al., 2001; Porter et al., 2007; Porter & Smithson, 2001, 2002), and Achieve (Resnick et 
al., 2004). These models differ significantly in their evaluation frameworks and dimensions, each possessing distinct 
characteristics and applications (Martone & Sireci, 2009). Among the three models, the Porter Alignment Index of 
the Survey of Enacted Curriculum (SEC) alignment model is a reliable indicator of improved student performance, 
displaying a high correlation with other alignment indicators (Porter & Smithson, 2021, 2002). Moreover, the cod- 
ing process used for Porter alignment analysis has been demonstrated to be highly reliable (Porter & Smithson, 
2021, 2002). Given the quantitative nature of this study and the need for acommon measurable standard, the SEC 
alignment analysis was chosen. 

The SEC model of alignment analysis was developed by Porter and Smithson in 2001. This model focuses on 
constructing a ‘content-cognitive level’ matrix to assess the degree of alignment between student achievement 
and classroom instruction (Blank et al., 2001; Polikoff et al., 2011). Alignment, according to Porter (2002), emerged 
as a core concept in educational reform centered around curriculum standards that transformed the education 
system and influenced curriculum resources, teachers’ professional development, and examinations and assess- 
ments (Blank et al., 2001; Polikoff et al., 2011; Porter & Smithson, 2001). 

Alignment analysis supported by the SEC model involves constructing a two-dimensional matrix represent- 
ing curriculum standards, classroom teaching, or academic assessment, with two dimensions: learning content 
and cognitive requirements. The alignment coefficients, denoted as P, are calculated by comparing the data with 
reference values obtained through statistical software to determine the degree of alignment (Porter & Smithson, 
2001; Porter, 2002). The alignment coefficient formula is as follows (1): 


p=1—= xl (1) 


In this formula, X represents specific target numbers in matrix 1, while Y represents specific target numbers 
in matrix 2. The Porter Alignment Index (P) ranges between 0 and 1, with values closer to 1 indicating greater 
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alignment and values closer to 0 indicating more significant deviation (Lu & Liu, 2012; Porter & Smithson, 2001). 
Research Questions 


This study employs the SEC analysis method to examine the alignment between the HSCCS and the 2018-2022 
NCEEs in the ‘Chemical Reaction Principles’ at the Chinese upper secondary level. The study aims to address the 
following three questions: 

1. What is the overall alignment between the HSCCS and the NCEEs? 

2. To what extent is there alignment between the HSCCS and the NCEEs in terms of themes of ‘Reaction 
Systems and Energy’, ‘Direction, Limits, and Rates of Chemical Reactions; and ‘lonic Reactions and 
Equilibria in Aqueous Solutions’? 

3. What is the level of alignment between the HSCCS and the NCEEs regarding cognitive level? 


Research Methodology 
The Development of the Alignment Analytical Framework 


In this study, the HSCCS (2017 Edition 2020 Revision) issued by the Ministry of Education of China (Ministry of 
Education of China, 2020) and the 2018-2022 NCEEs (National 1/B Version) were selected as the study materials. 
Specifically, the study focused on the ‘Chemical Reaction Principles’ section of upper-secondary school chemistry 
to help teachers clarify the direction and focus in teaching this section and, in turn, help students solve their long- 
standing difficulties in learning the ‘Chemical Reaction Principles’ To facilitate the analysis, an alignment analysis 
framework was designed based on the HSCCS and SEC analysis methods (Liu & Fulmer,2008; Porter, 2002). Based 
on the coding principles (Wei & Ou, 2019, Yu et al., 2022), the HSCCS and 2018-2022 NCEEs were then coded and 
transformed into two-dimensional matrixes: Theme and Cognitive Level. 


1. The Categories of Themes in the ‘Chemical Reaction Principles’ 


Because of the large number of knowledge points in this part of ‘Chemical Reaction Principles, the themes 
need to be divided appropriately and rationally. The concept of ‘Chemical Reaction Principles’ was divided into 
three dimensions and 11 themes according to the conceptual system of the HSCCS (Table 2) (Ministry of Education 
of China, 2020). HSCCS include 3 main themes, 11 sub-themes, and 40 knowledge points. 40 knowledge points 
were chosen as the basis for coding because they are more clearly and precisely expressed. 


Table 2 
Themes in the ‘Chemical Reaction Principles’ (Ministry of Education of China, 2020) 


Main Theme Sub-Theme 
Systems and Energy 


Reaction Systems and Energy Chemical Reactions and Thermal Energy 


Chemical Reactions and Electrical Energy 


Direction and Limits of Chemical Reactions 


Direction, Limits, and Rates of Chemical Reactions Chemical Reaction Rate 


Modulation of Chemical Reactions 


Behavior of Electrolytes in Aqueous Solutions 


lonization Equilibrium 


lonic Reactions and Equilibria in Aqueous Solutions Hydrolytic Equilibrium 


Precipitation Dissolution Equilibrium 


Application of lonic Reactions and Equilibria 
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2. The Identification of Cognitive Levels 


In this research, the 5 levels of cognitive level in the existing SEC analysis method (Porter, 2002; Porter & Smith- 
son, 2002) were adapted and reclassified into three levels according to the curriculum standards and the relevant 
research findings of subject experts (Ministry of Education of China, 2020; Motlhabane, 2017; Yu et al., 2022). The 
reason for the adaptation and classification based on the new curriculum was that: 

(1) Curriculum standards were developed by experts in the field based on national conditions 
and long-term research and were relatively authoritative and referable (Klein, 1999; Ministry of 
Education of China, 2020). 

(2) The requirements of the curriculum standards for each part of the subject matter were expressed 
as a combination of behavioral verbs and specific knowledge points, and specific behavioral 
verbs such as know, understand, or master were used to describe the level that students need to 
achieve (Ministry of Education of China, 2020), which could help researchers to make judgments 
about the cognitive level of each knowledge point in the curriculum standards. 

(3) Teachers mainly taught according to the cognitive levels required by the curriculum standards. 
They knew little about the 5 cognitive levels they were accustomed to using in the original SEC 
alignment analysis model; therefore, the localized modification of the cognitive levels accord- 
ing to the curriculum standards could more accurately reflect the teachers’ chemical reactions. 

According to the requirements of the new curriculum and the analysis and interpretation of experts, the 
cognitive learning objectives of NCEEs were divided into 3 different levels, each level corresponding to a specific 
descriptive verb, namely: ‘Level A’ ‘Level B’ and ‘Level C’ The behavioral verbs corresponding to each cognitive 
level are shown in Table 3 below. 


Table 3 
Cognitive Levels for Each Behavioral Verb of the Curriculum Standards 


Cognitive level Behavioral verb 
A Know, say, identify, describe, example, list 
B Understand, can express, identify, distinguish, and compare 
C Master, apply, explain, explain, predict, classify, summarize, and analyze 


3. SEC Alignment Analytical Framework for ‘Chemical Reaction Principles’ 
The alignment analytical framework based on the above coding methods was generated, as Table 4 shows. 


Table 4 
Alignment Analysis Framework 


Cognitive level 


F F Themes ————————___ Summation 
Dimensions A B Cc 
Systems and Energy 
Reaction Systems and Energy Chemical Reactions and Thermal Energy 


Chemical Reactions and Electrical Energy 


Direction and Limits of Chemical Reactions 


Direction, Limits, and Rates of Chemical Reactions Chemical Reaction Rate 


Modulation of Chemical Reactions 


554 


https://doi.org/10.33225/jbse/24.23.550 


Journal of Baltic Science Education, Vol. 23, No. 3, 2024 


ALIGNMENT BETWEEN CURRICULUM STANDARDS AND ASSESSMENT IN UNDERSTANDING ISSN 1648-3898 | jerinty 
CHEMICAL REACTION PRINGIPLES AT UPPER-SECONDARY SCHOOLS 
cig gai ISSN 2538-7138 joniine/ 


Cognitive level 


Di ‘ Themes ——————-__ Summation 
imensions A B c 


Behavior of Electrolytes in Aqueous Solutions 


lonization Equilibrium 


lonic Reactions and Equilibria in Aqueous Solutions Hydrolytic Equilibrium 


Precipitation Dissolution Equilibrium 


Application of lonic Reactions and Equilibria 


Summation 


Coding and Data Analysis 


Once the alignment analysis framework has been established, it is necessary to code the HSCCS and the 
2018-2022 NCEEs separately according to the framework and then analyze their alignment. The resulting content 
matrices were converted into ratio matrices to compare the degree of alignment between the HSCCS and the 
NCEEs. The study employed an overall analysis by calculating the Poter alignment coefficient, followed by a specific 
analysis of the two dimensions (themes and cognitive level). This approach aimed to evaluate the quality of the 
NCEEs and gain insights into the level of alignment with the curriculum standards. 


1. Coding and analysis of HSCCS 


The dimension‘Reaction Systems and Energy’ was coded as“1’,Direction, Limits, and Rates of Chemical Reac- 
tions’ was coded as “2”, etc. Under each dimension, some themes were coded as “1.1",“1.2", and “1.3”... E.g., the 
theme ‘Systems and energy’ under the first dimension was coded as “1.1”, and ‘Chemical Reactions and Thermal 
Energy’ was coded as “1.2” These themes were juxtaposed. There are specific knowledge points under the themes 
that were coded as “1.1.1” and “1.1.2”....... Based on the above coding method, three levels of numerical coding 
were formed in the thematic dimension. 

In addition, in the HSCCS, the objective descriptions of specific knowledge points are in the form of‘action verb 
+ knowledge point’ so the cognitive level of each knowledge point can be judged according to the cognitive level 
of the action verb. E.g., in the curriculum standard, there is such an objective as ‘The working principle of primary 
batteries and common chemical power sources, which corresponds to the behavior verb‘understand’ This behavior 
verb belongs to ‘level B’ so it is counted as “1” in the cell of ‘level B’ of ‘chemical reactions and electrical energy’. 

During the actual coding process, some problems were encountered. To conduct the study better, the follow- 
ing coding principles were established: 

(1) Ifthere are multiple knowledge points in the description of the same knowledge objective, coders 
should choose to code the knowledge points separately (Yu et al., 2022). For example, in‘'Under- 
stand the principles of salt hydrolysis and the main factors affecting salt hydrolysis; ‘The principles 
of salt hydrolysis, and ‘The main factors affecting salt hydrolysis’ were two knowledge points. In 
this case, the two knowledge points should be coded separately. Since both knowledge points 
belong to the theme ‘Chemical reactions and electrical energy’ and since they both correspond 
to a cognitive level of ‘Level B) they are counted as 2 on the ‘Level B’ cell of the two-dimensional 
content matrix ‘Chemical reactions and electrical energy’ 

(2) In the coding process, if a knowledge point contains more than one behavioral verb, the be- 
havioral verb with the highest cognitive level is selected to code the knowledge point (Wei & 
Ou, 2019). For example, in the expression of the knowledge point ‘Understand Hess law and its 
simple application’, the behavioral verbs for ‘Hess law’ include ‘understand (cognitive level B)’ 
and ‘apply (cognitive level C)’ in this case, according to the principle of ‘on high, not low’, ‘Hess 
law’ is coded as level C at the cognitive level, so the cell of level C in the content matrix ‘Chemical 
reactions and heat energy’ counts 1. 

(3) If a knowledge point contains only experiential target behavior verbs, the knowledge point 
should not be coded. 
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(4) If an objective statement contains multiple types of behavioral verbs, such as outcome and 
experiential, only the behavioral verbs of the outcome objective are considered. E.g.,‘To under- 
stand the effects of concentration, pressure, and temperature on chemical equilibrium through 
experimental investigation’ contains two verbs: ‘investigate’ and ‘understand’ In this case, only 
‘to understand’ is coded, and the process verb ‘to investigate’ is not coded. 

According to the above coding principles, two graduate students in the program coded the curriculum stan- 
dards separately. The Pearson correlation coefficient between the two coding results was calculated using SPSS 
at 0.985 (p = .01), indicating that the coding data of this study are stable and reliable. After thorough discussion 
and negotiation regarding the inconsistencies in the coding results between the two individuals, the final coding 
of the standards was determined (Table 5). To facilitate the subsequent alignment analysis, the two-dimensional 
content matrix criteria were transformed into a two-dimensional ratio matrix with the ratio integrated equal to 1. 


Table 5 
Two-Dimensional Content (Ratio) Matrix of The Curriculum Standards 


Cognitive level 


Dimensions Themes SS Summation 
A B Cc 
Systems and Energy 2 (0.050) 2 (0.050) 0 (0.000) 4 (0.100) 
Reaction System$ == Chemical Reactions and Thermal Energy 0 (0.000) 2 (0.050) 4 (0.025) 3 (0.075) 
and Energy 
Chemical Reactions and Electrical Energy 1 (0.025) 5 (0.125) 1 (0.025 7 (0.175) 
Direction and Limits of Chemical Reactions 3 (0.075) 3 (0.075) 0 (0.000 6 (0.150) 
Direction, Limits, and 
Rates of Chemical Chemical Reaction Rate 2 (0.050) 3 (0.075) 0 (0.000) 5 (0.125) 
Reactions 
Modulation of Chemical Reactions 1 (0.025) 1 (0.025) 0 (0.000 2 (0.050) 
Behavior of Electrolytes in Aqueous Solutions 0 (0.000) 1 (0.025) 0 (0.000) 1 (0.025) 
lonization Equilibrium 0 (0.000) 5 (0.125) 1 (0.025 6 (0.150) 
lonic reactions and 
equilibria in aqueous Hydrolytic Equilibrium 0 (0.000) 2 (0.050) 0 (0.000) 2 (0.050) 
luti 
ee Precipitation Dissolution Equilibrium 0 (0.000) 2 (0.050) 0 (0.000 2 (0.050) 
Application of lonic Reactions and Equilibria 0 (0.000) 0 (0.050) 2 (0.050) 2(0.050) 
Summation 9 (0.225) 26 (0.650) 5 (0.125) 40 (1.000) 


Note: Values in parentheses are ratio matrices, values outside parentheses are content matrices 


2. Coding and Analysis of NCEEs 


When coding the NCEEs, all the items involving the knowledge points of the‘Selective Compulsory 1: Chemical 
Reaction Principles’ were screened, and the solution process was also listed first. Then, according to the theme and 
cognitive level of the knowledge points involved in each step, the corresponding scores were filled into the cells of 
the corresponding two-dimensional content matrix, forming a two-dimensional matrix with scores as units, and then 
standardized into a two-dimensional ratio matrix. The analysis of the NCEE questions revealed that there are two 
types of questions involving chemistry: single-choice and subjective fill-in-the-blank questions. For single-choice 
questions, there are four options, sometimes involving different knowledge points, so when processing them, it is 
necessary to code each option separately and convert 6 points per question into one option, i.e., 1.5 points, to fill 
in the corresponding knowledge point and cognitive level two-dimensional framework. In addition, the subjective 
questions also need to be coded precisely to each blank of each sub-question. Similarly, if there is more than one 
knowledge point per blank, the marks of this blank will be equally distributed and filled into the two-dimensional 
matrix. The coding results (two-dimensional matrix) for the 2018-2022 NCEEs are shown in Appendix 1. 
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3. Alignment Comparison 


The data collected and coded were filled into the cells of the corresponding matrix to obtain the two-dimen- 
sional content matrices of the corresponding HSCCS and the NCEEs for a particular year. The alignment coefficient 
P was used to determine whether the HSCCS are effectively implemented in the NCEEs using the unadorned func- 
tion in MATLAB (Porter & Smithson, 2001; Porter, 2002). 

In addition, Porter’s alignment coefficient P only reflects the degree of alignment between the NCEEs and 
the HSCCS in general. However, it does not allow for an in-depth discussion and analysis of the specific alignment 
or deviation of the themes and cognitive levels of the two subjects (Porter & Smithson, 2001; Lu & Liu, 2012). The 
concept of ‘ratio differential’ was introduced to analyze the degree of deviation or alignment between any two 
variables at each theme or cognitive level (Yang, 2023; Yu et al., 2022). The formula for calculating the ratio differ- 
ence is as follows: 

AR=|R,-R, (2) 

R, represents the ratio of a theme to a cognitive level in the NCEE questions, and R, is the ratio of a theme to 
a cognitive level in the curriculum standards. R>O means that the ratio of a content topic or cognitive level in the 
NCEEs is higher than that required by the curriculum standards, and vice versa. AR represents the degree of devia- 
tion between these two variables, with larger values indicating a greater unalignment. 


Research Results 
Overall Alignment Results 


Bringing the ratio matrix of the 2018-2022 NCEEs and the ratio matrix of the HSCCS into the Porter alignment 
coefficient formula gives an overall picture of the alignment of the 2018-2022 NCEEs with the HSCCS. The Porter 
alignment coefficient formula is as follows: 


P=1- oI (3) 

(n denotes the number of cells in each ratio matrix. The previously defined alignment analysis framework 
results in an‘11*3’content matrix, so n equals 33 in this study; represents the value of cell iin the two-dimensional 
ratio matrix of the HSCCS; denotes the value of cell i of the NCEE rate matrix) 

The alignment coefficients for the 2018-2022 NCEEs and the HSCCS were obtained from the Porter coefficients 
coefficient formula above, as shown in Table 6. The P values range from 0 to 1, with higher values representing a 
higher degree of alignment between the NCEEs and the HSCCS. 


Table 6 
Porter Alignment Coefficient for 2018-2022 NCEEs and HSCCS 


Test Year 2018 2019 2020 2021 2022 
P 40 A5 A4 AT 52 


To more accurately study the degree of alignment between the 2018-2022 NCEEs and the HSCCS, the Unidrnd 
function in Matlab software was used to randomly assign the 40 knowledge points in the HSCCS and the marks 
of the ‘Chemical Reaction Principles’ in each year’s NCEE to two “113” two-dimensional content. The matrix was 
used to calculate the critical value of the alignment coefficient at the 0.05 significance level, i.e., the minimum 
acceptable level of alignment in this case. The threshold values for significant alignment between the HSCCS and 
the NCEEs in this study are shown in Table 7. Only when the alignment coefficient of the test is greater than the 
threshold value will the assessment be statistically significant and aligned with the standards. 
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Table 7 
Means, Standard Deviations, and Critical Values of P Under Normal Distribution 


Test Year M, SD, TH, Difference between P and the alignment threshold 
2018 A6 07 0.54 14 
2019 A6 07 0.59 14 
2020 AQ 07 0.60 16 
2021 A8 07 0.57 10 
2022 A2 07 0.57 05 


From the parameters related to the P-values in Table 7, it can be seen that there is no statistically significant 
alignment between the 2018-2022 NCEEs and the HSCCS in the ‘Chemical Reaction Principles. Among the five 
sets of assessments, the level of alignment between the NCEEs and the HSCCS is the highest in 2022, followed by 
2021, and the worst in 2020. In general, the alignment between the NCEEs and the HSCCS increased year by year 
with the advancement of the new curriculum reform, except for 2020. 


Alignment Result of Themes 


A comparison of the average distribution of the 2018-2022 NCEEs on each theme with the distribution of 
each theme required by the HSCCS is in Figure 1. 


Figure 1 
Difference Between the Average Rate of NCEEs in Each Content and The Rate of the HSCCS 
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Note: 1: Reaction Systems and Energy; 2: Direction, Limits, and Rates of Chemical Reactions; 3: lonic Reactions and Equilibria in 
Aqueous Solutions; 1.1: Systems and Energy; 1.2: Chemical Reactions and Thermal Energy; 1.3: Chemical Reactions and Electrical 
Energy; 2.1: Direction and Limits of Chemical Reactions; 2.2: Chemical Reaction Rate; 2.3: Modulation of Chemical Reactions; 3.1: 
Behavior of Electrolytes in Aqueous Solutions; 3.2: lonization Equilibrium; 3.3: Hydrolytic Equilibrium; 3.4: Precipitation Dissolu- 
tion Equilibrium; 3.5: Application of lonic Reactions and Equilibria 
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As can be seen from Figure 1, NCEE averages are higher than the requirements of the HSCCS for five themes: 
‘Chemical Reactions and Thermal Energy; ‘Chemical Reactions and Electrical Energy’, ‘Directions and Limits of 
Chemical Reactions; ‘Precipitation Dissolution Equilibria’ and ‘Applications of lonic Reactions and Equilibria, while 
the remaining six themes are lower than the requirements of the HSCCS. Across all themes, the absolute ratio dif- 
ferential for‘Systems and Energy; ‘Chemical Reactions and Electricity; ‘Directions and Limits of Chemical Reactions, 
‘lonization Equilibria and Applications of lonic Reactions and Equilibria’ were higher than 0.05, indicating that these 
five parts of the examination showed significant deviations from the HSCCS, with the largest deviations being in 
‘Systems and Energy’and ‘Application of lonic Reactions and Equilibria, with the absolute value of the ratio differ- 
ence between them exceeding 0.1. 


1. Dimension of ‘Chemical Reactions and Energy’ 


An analysis of the alignment of the dimension of ‘Chemical Reactions and Energy’ (Figure 2) shows that the 
theme with the highest degree of alignment is ‘Chemical Reactions and Thermal Energy’, and the distribution of 
NCEE each year does not vary much from the requirements of the HSCCS. The greatest overall difference was in 
the ‘Systems and Energy’, which was not examined in any of the five years’ NCEEs. For ‘Chemical Reactions and 
Electrical Energy; the actual NCEEs in 2019, 2021, and 2022 were less different from the HSCCS, while the NCEEs in 
2018 and 2020 exceeded the requirements of the curriculum standards. 


Figure 2 
Comparison of the Thematic Dimensions on ‘Chemical Reactions and Energy’ in the 2018-2022 NCEEs and HSCCS 


0,450 
0,400 
0,350 
0,300 
0,250 
0,200 
0,150 
0,100 Cee a 
0,050 vt 
0,000 


Ratio 


Systems and energy Chemical reactions and = Chemical reactions and 
thermal energy electrical energy 


Theme 


[==712018 2019 a 2020 2021 GHEE 2022 +*@+ Curriculum Standard 


2. Dimension of ‘Direction, Limits, and Rates of Chemical Reactions’ 


For dimensions of ‘Directions, Limits, and Rates of Chemical Reactions; as can be seen from Figure 3, the gap 
between the HSCCS and the NCEESs is relatively large compared to ‘chemical Reactions and Energy; and there are 
significant gaps for all three themes. For ‘Directions and Limits of Chemical Reactions; all five years of the NCEEs 
exceeded the requirements of the HSCCS, while for ‘Regulation of Chemical Reactions, the requirements of the 
HSCCS were largely missed. For ‘Rates of Chemical Reactions, the 2018 and 2019 NCEEs do not differ significantly 
from the HSCCS, while in the latter three years’ NCEEs, they are largely unexamined, which is a significant gap from 
the HSCCS. 
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Figure 3 
Comparison of the Thematic Dimensions on ‘Direction, Limits, and Rates of Chemical Reactions’ in the 2018-2022 NCEEs and 
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3. Dimension of‘lonic Reactions and Equilibria in Aqueous Solutions’ 


For dimensions of ‘lonic Reactions and Equilibria in Aqueous Solutions, there were relatively large differences 
between the NCEEs and the HSCCS (Figure 4). For the‘lonic Reactions and Equilibria in Aqueous Solutions, the five- 
year NCEEs were generally above the requirements of the HSCCS, while for the other four themes, the NCEEs did 
not meet the requirements of the HSCCS in all years except for a few years when the requirements of the HSCCS 
were slightly exceeded. 


Figure 4 
Comparison of the Thematic Dimensions on ‘lonic Reactions and Equilibria in Aqueous Solutions’ in the 2018-2022 NCEEs 
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Alignment Result of Cognitive Levels 


The mean ratio differential between the HSCCS and the distribution of the last five-year NCEEs for each cogni- 
tion level were also analyzed. Overall, there is a wide gap between the average examination of students’ cognitive 
levels in the recent 5-year NCEEs and the HSCCS, with the absolute ratio difference between the two being above 
0.05 (Figure 5). A relatively good alignment is found at Level B, followed by Level A, with Level C showing the poor- 
est. At Level C, the NCEEs were tested well above the requirements of the HSCCS. 


Figure 5 
Difference Between the Average Examination Rate of NCEEs and the HSCCS Requirement Rate for Each Cognitive Level 
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Further Analysis of the differences in cognitive level of examination between the NCEEs and the HSCCS for 
each year was carried out, and the results are shown in Figure 6. 


Figure 6 
HSCCS Requirement and Percentage Distribution of 2018-2022 NCEEs Across Cognitive Levels 
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As can be seen from Figure 6, at the ‘Level A’ cognitive level, the 2019 NCEE has the highest percentage of 
questions at 0.161, which is also the most consistent with the curriculum standard but still slightly below the cur- 
riculum standard of 0.225, while the 2020 NCEE has the lowest percentage of questions at 0.029. At the ‘Level B’ 
cognitive level, the 2022 NCEE has the highest percentage of questions at 0.745, and the 2020 NCEE also has a high 
percentage at 0.696. The remaining three years’ NCEEs are at a rate lower than the curriculum standard, with the 2021 
NCEE being the lowest at 0.404. At cognitive level C, the 2021 NCEE is the most tested at a rate of 0.456, followed 
by the 2018 NCEE at 0.381. The 2022 NCEE is the smallest, with a ratio of 0.182. In addition, 2018-2022 NCEEs all 
tested the‘Chemical Reaction Principles’ section at a ratio exceeding the curriculum standards for cognitive level C. 


Discussion 


Using the SEC alignment analysis method, this study coded the HSCCS and the 2018-2022 NCEEs into six 
matrixes and compared their alignment levels from the dimensions of overall alignment, content alignment, and 
cognitive level alignment. 

For the overall alignment level, the findings of this study indicate that there is a lack of significant alignment 
between the HSCCS and the NCEEs in China. This is consistent with previous research findings, which have also 
shown similar misalignment issues between the NCEEs, textbooks, and curriculum standards in the field of biology 
in China (Lu & Liu, 2012; Sun & Li, 2021; Yu et al., 2022). This study extends this issue to the field of chemistry and 
further confirms the existence of such misalignment problems in multiple subjects. This may result from several 
reasons. Firstly, the examination system may emphasize the assessment of student’s knowledge, while educational 
objectives of standards may focus on cultivating students’ comprehensive qualities and practical application abilities 
(Hong et al., 2019). This may cause some differences between the examination system and educational objectives. 
Secondly, the updating of textbooks and curriculum standards requires time and resources, while the development 
of examination questions may be faster and more flexible (Solomon, 2009). This can also result in misalignment 
between examinations and the latest educational requirements. Additionally, because of time, resources, and 
other limitations, the examination may have difficulties in comprehensively covering all aspects of the educational 
curriculum, thereby leading to misalignment with curriculum standards (Weir, 2005). 

Regarding the year comparison, the study also finds the overall alignment between the HSCCS and NCEEs 
varies over time. Although the overall alignment levels are not high, the alignment between the two has progres- 
sively improved as the years have passed. This can be attributed to the curriculum reforms undertaken in China in 
recent years, as well as the revisions made to the curriculum standards based on the lessons learned from these 
reforms (Yao & Guo, 2018). The latest version of the curriculum standards was issued in 2017 and revised in 2020 
(Ministry of Education of China, 2020). The 2018 NCEE was the first examination conducted after the release of 
the new curriculum standards, and it is possible that question writers may not have been fully acquainted with 
the updated standards at that time (Liang & Yuan, 2008; Liu et al., 2009; Lu & Liu, 2012). However, since then, the 
NCEEs have been gradually aligning with the curriculum standards, except for a possible deviation in 2020, which 
suggests the success of the Chinese national training for NCEE question proposers. In the future, it is crucial to 
continue providing training activities in various areas to ensure that question proposers have a deep understand- 
ing of the curriculum standards, can accurately assess students’ knowledge, and improve the quality of the NCEEs 
(Flowers et al., 2006; Lu & Liu, 2012). 

For the comparison of the content level, the average distribution of the five-year NCEE questions is closer to 
the curriculum standard for the theme of ‘Behavior of electrolytes in aqueous solutions, with only a minimal abso- 
lute difference of 0.003. On the other hand, the biggest differences were observed in the themes of ‘Application of 
ionic reactions and equilibrium’ and ‘Systems and energy; with absolute ratio differences exceeding 0.1. The NCEEs 
fell significantly short of the curriculum standard requirements for ‘Systems and energy’ while the examination 
questions for‘lonic reactions and applications of equilibrium’ went beyond the curriculum standard requirements. 
Moreover, the study categorized the themes of each dimension into three categories and further analyzed the 
specific gaps between the questions of each year and the curriculum standards. This analysis provides valuable 
insights for future improvements in the quality of examination questions. 

In terms of cognitive levels, there are also significant differences in the distribution between the NCEEs and 
the HSCCS. Compared to the curriculum standards, the NCEEs tend to focus more on higher cognitive levels while 
neglecting a comprehensive assessment of students’ foundational knowledge. Previous research also found the 
same results in mathematics education (Hong et al., 2019). This may be attributed to the purpose and role of the 
NCEEs. As a college entrance examination, the NCEEs aim to evaluate students’ comprehensive abilities and ap- 
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plication skills at a higher level, selecting outstanding talents (Wang, & Rao, 2022). Therefore, the examination 
questions are often designed to emphasize higher cognitive skills such as analysis, synthesis, and evaluation (Zou, 
2018). In contrast, the curriculum standards place more emphasis on students’ mastery of foundational knowledge 
and core concepts, laying a solid foundation for their future learning and career development. 


Conclusions and Implications 


Using the coded data from the curriculum standards and the 2018-2022 NCEEs as the sample of the study, the 
knowledge involving ‘Chemical Reaction Principles’ was divided into two dimensions, vertical in terms of themes 
and horizontal in terms of cognitive level, to construct an alignment analysis framework, and the alignment be- 
tween the two was analyzed, resulting in the following conclusions. The‘Chemical Reaction Principles’ section had 
no statistically significant alignment between the NCEEs and the curriculum standards. The NCEEs differed signifi- 
cantly from the curriculum standards in terms of themes, on average, being more consistent for‘ionic reactions and 
equilibria in aqueous solutions’; the examination of ‘Application of ionic reactions and equilibria’and‘Systems and 
energy’ deviated significantly from the curriculum standard, falling far short of or exceeding the requirements of 
the curriculum standard. The examination of each dimensional topic was not well balanced from year to year. The 
chemistry NCEEs also varied relatively markedly regarding cognitive level with the curriculum standard. Compared 
with the curriculum standards, the NCEEs have raised the cognitive demands on students. 

The findings of this study have implications for the field of chemistry education. Firstly, the alignment analysis 
framework used in this study provides theoretical references for future alignment studies in other disciplines and 
fields. The methodology and insights gained from this study can be applied to examine the alignment between 
examination questions and curriculum standards in subjects beyond chemistry. This expands the potential impact 
of the study and contributes to the broader field of educational research. Secondly, the results of this study serve 
as a reference for future reforms in classroom teaching and examination practices. By aligning the examination 
questions with the curriculum standards, educators can ensure that the teaching and assessment methods are 
more closely aligned with the requirements of the curriculum. This alignment will help create a more effective 
learning environment for students, meeting their cognitive needs and promoting a deeper understanding of 
the subject matter. Thirdly, for teachers, these findings guide teachers in terms of the direction and focus of their 
teaching. By understanding the gaps and inconsistencies between the examination questions and curriculum 
standards, teachers can better prioritize and emphasize critical knowledge and concepts in their teaching. This 
will enhance the quality of instruction and improve student learning outcomes. Finally, for students, the findings 
of this study benefit them directly by helping them master the relevant critical knowledge more effectively. By 
aligning the examination questions with the curriculum standards, students will have a clearer understanding of 
what is expected of them and can focus their efforts on acquiring the essential knowledge and skills. This align- 
ment will ultimately contribute to improved student performance and achievement. In summary, this study informs 
classroom teaching and examination reforms, guiding teachers in their instructional practices, enhancing student 
mastery of critical knowledge, and providing theoretical references for alignment studies in other disciplines. These 
implications contribute to the advancement of chemistry education and have broader implications in the field of 
educational research. 


Limitations and Future Study 


This study has certain limitations. Firstly, due to time and resource constraints, the study only focused on the 
module “Chemical Reaction Principles” in high school chemistry and did not analyze other modules. Therefore, 
future research will address these limitations and explore other modules in chemistry to gain a more comprehen- 
sive understanding of the gaps and inconsistencies between examination questions and curriculum standards. By 
doing so, researchers can provide valuable insights for future reforms of examination questions, ensuring that the 
questions align more closely with the curriculum standards and promote a deeper understanding of the subject 
matter. Additionally, this study only focused on the analysis of standards and entrance examination questions, 
without extending to other elements of the education system for a comprehensive alignment assessment. Future 
research will expand the study of alignment to other essential elements of the education system, such as high 
school chemistry curriculum standards, textbooks, and teachers’ teaching methods, to conduct a comprehensive 
and systematic analysis of alignment. By analyzing the alignment between these elements, researchers can examine 
the curriculum from a broader perspective and identify areas that require further reform. 
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Appendix A 2018-2022 NCEEs Coding Results (2-D Matrix) 


2018 NCEE Content (Rate) Matrix 


Content encoding (ratio value) 


Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
14 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
1 1.2 0(0.000) 0(0.000) 2(0.095) 2(0.095) 
1.3 0(0.000) 4(0.190) 4.5(0.214) 8.5(0.405) 
2.1 0(0.000) 5(0.238) 0(0.000) 5(0.238) 
2 22 3(0.143) 4(0.048) 0(0.000) 4(0.190) 
23 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.1 0(0.000 0(0.000) 0(0.000) 0(0.000) 
3.2 0(0.000 0(0.000) 0(0.000) 0(0.000) 
3 3.3 0(0.000 0(0.000) 0(0.000) 0(0.000) 
34 0(0.000 0(0.000) 0(0.000) 0(0.000) 
35 0(0.000 0(0.000) 1.5(0.071) 1.5(0.071) 
Summation 3(0.143 10(0.476) 8(0.381) 21(1.000) 
2019 NCEE Content (Rate) Matrix 
Content encoding (ratio value) 
Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
1 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
1 1.2 0(0.000) 1(0.032) 1(0.032) 2(0.065) 
13 0(0.000) 4.5(0.145) 0(0.000) 4.5(0.145) 
24 0(0.000) 7(0.226) 0(0.000) 7(0.226) 
2 22 5(0.161) 0(0.000) 0(0.000) 5(0.161) 
23 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.1 0(0.000) 3(0.097) 0(0.000) 3(0.097) 
3.2 0(0.000) 2(0.065) 0(0.000) 2(0.065) 
3 3.3 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.4 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.5 0(0.000) 0(0.000) 7.5(0.242) 7.5(0.242) 
Summation 5(0.161) 17.5(0.565) 8.5(0.274) 31(1.000) 
2020 NCEE Content (Rate) Matrix 
Content encoding (ratio value) 
Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
1 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
1 1.2 0(0.000) 0(0.000) 2(0.058) 2(0.058) 
1.3 0(0.000) 11(0.319) 0(0.000) 11(0.319) 
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Content encoding (ratio value) 


ISSN 1648-3898 | /print/ 
ISSN 2538-7138 /oniines 


Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
2.1 1(0.029) 7(0.203) 0(0.000) 8(0.232) 
2 22 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
23 0(0.000) 2(0.058) 0(0.000) 2(0.058) 
3.1 0(0.000) 0.5(0.014) 0(0.000) 0.5(0.014) 
3.2 0(0.000) 1.5(0.043) 0(0.000) 1.5(0.043) 
3 3.3 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.4 0(0.000) 2(0.058) 0(0.000) 2(0.058) 
3.5 0(0.000) 0(0.000) 7.5(0.217) 7.5(0.217) 
Summation 1(0.029) 24(0.696) 9.5(0.275) 34.5(1.000) 
2021 NCEE Content (Rate) Matrix 
Content encoding (ratio value) 
Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
1 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
1 1.2 0(0.000) 2(0.070) 1(0.035) 3(0.105) 
13 0(0.000) 0(0.000) 4.5(0.158) 4.5(0.158) 
2.1 3(0.105) 4(0.140) 0(0.000) 7(0.246) 
2 22 4(0.035) 0(0.000) 0(0.000) 1(0.035) 
23 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.1 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
3.2 0(0.000) 1.5(0.053) 0(0.000) 1.5(0.053) 
3 3.3 0(0.000) 4(0.035) 0(0.000) 1(0.035) 
34 0(0.000) 3(0.105) 0(0.000) 3(0.105) 
35 0(0.000) 0(0.000) 7.5(0.263) 7.5(0.263) 
Summation 4(0.140) 11.5(0.404) 13(0.456) 28.5(1.000) 
2022 NCEE Content (Rate) Matrix 
Content encoding (ratio value) 
Theme 
Cognitive level A Cognitive level B Cognitive level C Summation 
1 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
1 1.2 0(0.000) 0(0.000) 3(0.109) 3(0.109) 
1.3 0(0.000) 6(0.218) 0(0.000) 6(0.218) 
2.1 2(0.073) 6(0.218) 0(0.000) 8(0.291) 
2 2.2 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
23 0(0.000) 0(0.000) 0(0.000) 0(0.000) 
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Theme 


3.1 
3.2 


3.4 
3.5 


Summation 


Content encoding (ratio value) 


Cognitive level A Cognitive level B Cognitive level C Summation 
0(0.000) 0(0.000) 0(0.000) 0(0.000) 
0(0.000) 4.5(0.164) 0(0.000) 4.5(0.164) 
0(0.000) 0(0.000) 0(0.000) 0(0.000) 
0(0.000) 4(0.145) 0(0.000) 4(0.145) 
0(0.000) 0(0.000) 2(0.073) 2(0.073) 
2(0.073) 20.5(0.745) 5(0.182) 27.5(1.000) 
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